Automated Cross-prompt Scoring of Essay Traits
نویسندگان
چکیده
The majority of current research in Automated Essay Scoring (AES) focuses on prompt-specific scoring either the overall quality an essay or with regards to certain traits. In real-world applications obtaining labelled data for a target prompt is often expensive unfeasible, requiring AES system be able perform well when predicting scores essays from unseen prompts. As result, some recent has been dedicated cross-prompt AES. However, this line thus far only concerned holistic, scoring, no exploration into different users systems require feedback aspects their writing, trait necessary component effective system. Therefore, address need, we introduce new task named Cross-prompt Traits, which requires model trained solely non-target-prompt and predict score as number specific traits target-prompt essays. This challenges model's ability generalize order novel domain its represent multiple aspects. addition, new, innovative approach builds top state-of-the-art method Our utilizes trait-attention mechanism multi-task architecture that leverages relationships between each simultaneously individual trait. We conduct extensive experiments widely used ASAP ASAP++ datasets demonstrate our outperform leading methods.
منابع مشابه
Beyond Automated Essay Scoring
The ability to communicate in natural language has long been considered a defining characteristic of human intelligence. Furthermore, we hold our ability to express ideas in writing as a pinnacle of this uniquely human language facility—it defies formulaic or algorithmic specification. So it comes as no surprise that attempts to devise computer programs that evaluate writing are often met with ...
متن کاملAutomated Essay Scoring for Swedish
We present the first system developed for automated grading of high school essays written in Swedish. The system uses standard text quality indicators and is able to compare vocabulary and grammar to large reference corpora of blog posts and newspaper articles. The system is evaluated on a corpus of 1 702 essays, each graded independently by the student’s own teacher and also in a blind re-grad...
متن کاملAutomated Essay Scoring 1 Automated Essay Scoring: Writing Assessment and Instruction
Automated Essay Scoring 2 Introduction This chapter documents the advent and rise of automated essay scoring (AES) as a means of both assessment and instruction. The first section discusses what AES is, how it works, and who the major purveyors of the technology are. The second section describes outgrowths of the technology as it applies to ongoing projects in measurement and education. In 1973...
متن کاملAutomated Essay Scoring Using Machine Learning
We built an automated essay scoring system to score approximately 13,000 essay from an online Machine Learning competition Kaggle.com. There are 8 different essay topics and as such, the essays were divided into 8 sets which differed significantly in their responses to the our features and evaluation. Our focus for this essay grading was the style of the essay, which is an extension on the stud...
متن کاملAutomated Essay Scoring for Nonnative English Speakers
The e-rater system 1 , an automated essay scoring system, developed at Educational Testing Service (ETS), is currently being used to score essay responses on the Graduate Management Admissions Test (GMAT). The average agreement between human readers, and between independent human readers and e-rater is approximately 92%. There is much interest in the larger writing community in examining the sy...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2021
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v35i15.17620